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1 EXECUTIVE SUMMARY 


During the post-launch Cal/Val Phase of SMAP there are two objectives for each science product team: 1) 
calibrate, verify, and improve the performance of the science algorithms, and 2) validate accuracies of the 
science data products as specified in the LI science requirements according to the Cal/Val timeline. This 
report provides analysis and assessment of the SMAP Level 4 Carbon (L4_C) product specifically for the 
beta release. The beta-release version of the SMAP L4_C algorithms utilizes a terrestrial carbon flux 
model informed by SMAP soil moisture inputs along with optical remote sensing (e.g. MODIS) 
vegetation indices and other ancillary biophysical data to estimate global daily NEE and component 
carbon fluxes, particularly vegetation gross primary production (GPP) and ecosystem respiration (R eco )- 
Other L4_C product elements include surface (<10 cm depth) soil organic carbon (SOC) stocks and 
associated environmental constraints to these processes, including soil moisture and landscape FT 
controls on GPP and R eco (Kimball et al. 2012). The L4_C product encapsulates SMAP carbon cycle 
science objectives by: 1) providing a direct link between terrestrial carbon fluxes and underlying 
freeze/thaw and soil moisture constraints to these processes, 2) documenting primary connections 
between terrestrial water, energy and carbon cycles, and 3) improving understanding of terrestrial carbon 
sink activity in northern ecosystems. 

There are no LI science requirements for the L4_C product; however self-imposed requirements 
have been established focusing on NEE as the primary product field for validation, and on demonstrating 
L4_C accuracy and success in meeting product science requirements (Jackson et al. 2012). The other 
L4_C product fields also have strong utility for carbon science applications; however, analysis of these 
other fields is considered secondary relative to primary validation activities focusing on NEE. The L4_C 
targeted accuracy requirements are to meet or exceed a mean unbiased accuracy (ubRMSE) for NEE of 
1.6 g C m 2 d" 1 or 30 g C m 2 yr' 1 , emphasizing northern (>45°N) boreal and arctic ecosystems; this is 
similar to the accuracy level of tower eddy covariance measurement-based observations (Baldocchi 
2008). 

Methods used for L4_C performance and validation assessments include: 1) qualitative evaluations 
of product fields in relation to characteristic spatial and seasonal patterns; 2) comparisons of daily carbon 
flux estimates with in situ tower eddy covariance measurement-based daily carbon (CO 2 ) flux 
observations from core tower validation sites [CVS]; 3) comparisons of daily carbon flux estimates with 
more extensive historical tower carbon flux observations from global FLUXNET data archives; and 4) 
consistency checks against other synergistic global carbon products, including soil carbon inventory 
records, satellite-based productivity (GPP) records, global GPP records derived from tower observation 
upscaling methods, and satellite-based observations of solar induced canopy fluorescence (SIF) as a 
surrogate for net photosynthesis and GPP. The above CVS comparisons involve approximately 30 
individual tower sites; 12 of these sites emphasize northern (>45°N) ecosystems, which are a primary 
focus of the L4_C product science objectives, while 18 sites represent a diverse range of other global 
biome types. The CVS comparisons involve active participation from SMAP tower validation partners 
who have agreed to contribute near real-time tower observation data records. A larger set of historical 
tower observation records from 228 globally distributed sites was also used for L4_C validation and was 
provided by the FLUXNET La Thuile tower data synthesis (Baldocchi 2008); these data extend over 
multiple (2-7) years and were used to establish climatological records for each site, including daily means 
and variability (standard deviation, SD). The above analyses exceed criteria established by the Committee 
on Earth Observing Satellites (CEOS) for Stage 1 validation, which supports beta release of the data 
based on a limited set of core validation sites. The above activities also satisfy criteria for Stage 2 
validation by expanding to regional and global assessments that involve a diverse set of independent 
observations. 

The primary methods and metrics used for the L4_C Cal/Val assessment include comparisons of 
collocated time series plots of tower observations with L4_C daily outputs. Comparisons involving CVS 
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sites are both spatially and temporally consistent, while comparisons using the more extensive FLUXNET 
tower site records are spatially but not temporally consistent as they involve product evaluations against 
historical tower observations. Other methods employed for L4_C evaluations include qualitative 
comparisons of latitudinal means and spatial distributions between L4_C outputs and similar spatially 
contiguous climatological variables derived from other independent satellite, inventory and model-based 
products. Metrics used to evaluate relative agreement between L4_C product fields and the independent 
observations include correlation (r- value), RMSE differences, bias and model sensitivity diagnostics. The 
metrics used to evaluate L4_C NEE correspondence and target accuracy requirements for product success 
primarily focus on bias-adjusted (primary) results, but also include secondary assessments of the 
unadjusted results. 

This report notes several limitations in the beta-release version of the L4_C product, including the 
use of GEOS-5 surface temperatures rather than SMAP Radar-defined FT constraints to estimated carbon 
fluxes. These limitations will be addressed in the coming year prior to release of the validated data 
product. In addition, the validated product release will include more extensive validation activities 
involving a longer operational data record and associated calibration refinements; more detailed model 
sensitivity studies; and validation assessments using observational data records from several intensive 
field experiments. Despite these remaining areas, the beta-release L4_C product is of sufficient level of 
maturity and quality that it can be approved for distribution to and used by the larger science and 
application communities. This beta release also presents an opportunity to enable users to gain familiarity 
with the parameters and data formats of the product prior to full validation. 

2 OBJECTIVES OF CAL/VAL 

During the post-launch Cal/Val (Calibration/Validation) Phase of SMAP there are two objectives for each 
science product team: 

• Calibrate, verify, and improve the performance of the science algorithms, and 

• Validate accuracies of the science data products as specified in the Level 1 (LI) science 
requirements according to the Cal/Val timeline. 

The process is illustrated in Figure 2.1. In this Assessment Report the progress of the L4_C team in 
addressing these objectives for beta release is described. The approaches and procedures utilized follow 
those described in the SMAP Cal/Val Plan [Jackson et al. 2012] and Algorithm Theoretical Basis 
Document for the Level 4 Carbon Data Product [Kimball et al. 2012], 
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Post-launch Cal/Val Methodologies/Resources: Core validation sites, networks, 
model and satellite products, field experiments, analyses and assessments 


Figure 2.1. Overview of the SMAP Cal/Val Process. 


SMAP established a unified definition base in order to effectively address the mission requirements. 
These are documented in the SMAP Handbook/ Science Terms and Definitions [Entekhabi et al. 2014], 
where Calibration and Validation are defined as follows: 

• Calibration-. The set of operations that establish, under specified conditions, the relationship 
between sets of values or quantities indicated by a measuring instrument or measuring system and 
the corresponding values realized by standards. 

• Validation: The process of assessing by independent means the quality of the data products 
derived from the system outputs. 

The L4_C product does not have a documented L 1 accuracy requirement; instead the L4_C team adopted 
a self-imposed accuracy requirement threshold of 1.6 g C m 2 d" 1 or 30 g C m 2 yr' 1 (RMSE) for the bias- 
adjusted model NEE outputs, emphasizing northern (>45°N) ecosystems, and at the level of observation 
uncertainty from tower eddy covariance monitoring sites (Baldocchi 2008). 

In order to ensure the public’s timely access to SMAP data, before releasing validated products the 
mission is required to release beta-quality products. The maturity of the products in the beta release is 
defined as follows: 

• Early release is used to gain familiarity with data formats. 

• The beta release is intended as a testbed to discover and correct errors. 

• The beta release is minimally validated and still may contain significant errors. 

• The general research community is encouraged to participate in the quality assessment and 
validation, but need to be aware that product validation and quality assessment (QA) are ongoing. 

• Data may be used in publications as long as the fact that it is beta quality is indicated by the 
authors. Drawing quantitative scientific conclusions is discouraged. Users are urged to contact 
science team representatives prior to use of the data in publications, and to recommend members 
of the instrument teams as reviewers. 

• The estimated uncertainties will be documented. 


6 





• The beta release data may be replaced in the archive when an upgraded (provisional or validated) 
product becomes available. 

Due to the initially favorable quality of the SMAP observations and Level 4 model data assimilation 
system, this beta release of the L4_C product is closer to a provisional release, which is defined as: 

• Incremental improvements are ongoing. Obvious artifacts or errors observed in the beta product 
have been identified and either minimized or documented. 

• General research community is encouraged to participate in the QA and validation, but need to be 
aware that product validation and QA are ongoing. 

• Product may be used in publications as long as provisional quality is indicated by the authors. 
Users are urged to contact science team representatives prior to use of the data in publications, 
and to recommend members of the instrument teams as reviewers. 

• The estimated uncertainties will be documented. 

• The provisional release data will be replaced in the archive when an upgraded (validated) product 
becomes available. 

In assessing the maturity of the L4_C product, the L4_C team also considered the guidance provided 
by the Committee on Earth Observation Satellites (CEOS) Working Group on Calibration and Validation 
(WGCV): 

• Stage 1 : Product accuracy is assessed from a small (typically < 30) set of locations and time 
periods by comparison with in situ or other suitable reference data. 

• Stage 2: Product accuracy is estimated over a significant set of locations and time periods by 
comparison with reference in situ or other suitable reference data. Spatial and temporal 
consistency of the product and with similar products has been evaluated over globally 
representative locations and time periods. Results are published in the peer-reviewed literature. 

• Stage 3: Uncertainties in the product and its associated structure are well quantified from 
comparison with reference in situ or other suitable reference data. Uncertainties are characterized 
in a statistically robust way over multiple locations and time periods representing global 
conditions. Spatial and temporal consistency of the product and with similar products has been 
evaluated over globally representative locations and periods. Results are published in the peer- 
reviewed literature. 

• Stage 4: Validation results for stage 3 are systematically updated when new product versions are 
released and as the time-series expands. 

For the beta release the L4_C team’s Stage 1 and Stage 2 (global assessment) activities have been 
established and are relatively mature. These Cal/Val program activities will continue toward validated 
release, including analyses of longer data records and updates from planned calibration refinements. 

These activities will continue through all Cal/Val stages over the SMAP mission life span. 

3 EXPECTED L4_C ALGORITHM AND PRODUCT 
PERFORMANCE 

The L4_C algorithm performance, including variance and uncertainty estimates of model outputs, was 
determined during the mission pre-launch phase through spatially explicit model sensitivity studies using 
available model inputs similar to those currently being used for operational production and evaluating the 
resulting model simulations over the observed range of northern (>45 °N) and global conditions (Kimball 
et al. 2012, Entekhabi et al. 2014). The L4_C algorithm options were also evaluated during the mission 
prelaunch phase, including deriving canopy fPAR (fraction of photosynthetically active radiation 
absorbed by the canopy) from lower order NDVI (Normalized Difference Vegetation Index) inputs in lieu 
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of using MODIS (MOD15) fPAR; and including an explicit model representation of boreal fire 
disturbance recovery impacts. These results indicated that the L4_C accuracy requirements (i.e. NEE 
RMSE < 30 g C m' 2 yr' 1 ) could be met from the baseline algorithms over more than 82% and 89% of 
global and northern vegetated land areas, respectively (Yi et al. 2013, Kimball et al. 2012). 

The global L4_C algorithm error budget for NEE derived during the mission prelaunch phase 
indicated that the estimated NEE RMSE uncertainty is proportional to GPP and is therefore larger in 
higher biomass productivity areas, including forests and croplands. Likewise, NEE RMSE uncertainty is 
expected to be lower in less productive areas, including grasslands and shrublands. Expected model NEE 
RMSE levels were also generally within targeted accuracy levels (NEE RMSE <30 g C m 2 yr' 1 ) for 
characteristically less productive boreal and Arctic biomes, even though relative model error as a 
proportion of total productivity (NEE RMSE / GPP) may be large in these areas. The estimated NEE 
uncertainty was lower than expected in some warmer tropical high biomass productivity areas (e.g. 
Amazon rainforest) because of reduced low temperature and moisture constraints to the L4_C respiration 
calculations so that the bulk of model uncertainty is contributed by GPP in these areas. Model NEE 
uncertainty in the African Congo was estimated to be relatively larger than in Amazonia due to relatively 
drier climate conditions in central Africa and associated larger uncertainty contributions of soil moisture 
and temperature inputs to the model respiration and GPP calculations. 

4 L4_C PROCESSING OPTIONS 

The current beta-release L4_C baseline product reflects various processing options that are implemented 
in the algorithm preprocessing stage for handling of the daily model inputs. These processing options are 
distinct from other options that are more internal to the model algorithms (Kimball et al. 2012). Two 
major preprocessing options are used in the L4_C beta-release product, namely, the use of estimated 
clear-sky fPAR inputs for missing or lower quality MODIS fPAR inputs, and the use of GMAO surface 
temperature fields to estimate frozen temperature constraints to the GPP calculations instead of SMAP FT 
defined constraints. The use of these preprocessing options are noted in the L4_C product bit flags as 
defined in the product specification document (Glassy et al. 2015). 

The preprocessing options used in the beta-release product include a grid cell-wise selection of a 
MODIS fPAR 8-day climatology value where there data quality flag information from the operational 
MODIS fPAR inputs indicate missing or lower quality (QC) cloud contaminated data. The static MODIS 
global fPAR climatology is part of the ancillary data used for L4_C processing and was derived on a per 
grid-cell basis as the mean fPAR value for each 8-day time step over an annual cycle as determined from 
the best QC MODIS MOD 15 fPAR long-term (2000-2012) record. The spatial extent of the global 
MODIS fPAR climatology also defines the global L4_C product domain. The use of the fPAR screening 
process and climatology generally improves model performance, especially in areas with persistent cloud 
cover, including the tropics and northern boreal/ Arctic ecosystems. However, frequent substitution of 
current fPAR retrievals for alternative climatological values established from a long-term historical record 
may degrade model sensitivity to seasonal and annual climate variations, impacts for recent climate trends 
and extreme events, and recent land use and land cover changes. The fPAR quality bit flag information in 
the L4_C product provides a record of the spatial distribution and temporal frequency of these 
substitutions and facilitates future studies to evaluate these impacts. 

The L4_C beta-product includes the use of a land surface temperature-defined FT frozen flag to 
define frozen temperature constraints to the model GPP calculations (Kimball et al. 2012). The FT frozen 
temperature flag is obtained from the lower order SMAP L3_SM_A inputs when available; when the 
L3_SM_A inputs are missing the FT frozen flag information is obtained from similar daily surface 
temperature inputs from the GMAO GEOS-5 land model, where temperatures below a 0.0°C threshold 
are defined as frozen. The major impact of using temperature-defined frozen flags from the land model is 
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that the FT flags are derived from relatively coarse simulations that are not directly informed by SMAP 
observations. Future L4_C product releases will benefit from FT frozen constraints defined from SMAP 
microwave retrievals with enhanced L-band sensitivity to landscape FT dynamics. These updates are 
expected to have the greatest benefit in northern ecosystems with greater frequency of frozen conditions, 
and in complex terrain and during seasonal FT transitions with larger FT spatial heterogeneity (Du et al. 
2014). The use of SMAP FT inputs will also enhance SMAP carbon cycle science objectives 
encapsulated by the L4_C product, including improving understanding of the net carbon sink in boreal 
ecosystems (Entekhabi et al. 2010). 


5 APPROACH FOR L4_C CAL/VAL: METHODOLOGIES 

Validation is critical for accurate and credible product usage and must be based on quantitative estimates 
of uncertainty. For satellite-based retrievals, validation should include direct comparison with 
independent correlative measurements. The assessment of uncertainty must also be conducted and 
presented to the community in normally used metrics in order to facilitate acceptance and 
implementation. 

During the mission definition and development period, the SMAP Science Team and Cal/Val 
Working Group identified the metrics and methodologies that would be used for L2-L4 product 
assessment. These metrics and methodologies were vetted in community Cal/Val Workshops and tested 
in SMAP pre-launch Cal/Val rehearsal campaigns. The methodological elements identified and their 
general roles are: 

1. Core Validation Sites: Accurate estimates of products at matching scales for a limited set of 
conditions 

2. Sparse Networks: One point in the grid cell for a wide range of conditions 

3. Satellite Products: Estimates over a very wide range of conditions at matching scales 

4. Model Products: Estimates over a very wide range of conditions at matching scales 

5. Field Campaigns: Detailed estimates for a very limited set of conditions 

In the case of the L4_C data product, all of the above elements can contribute to product assessment 
and improvement. With regard to the CEOS Cal/Val stages, Core Validation Sites address Stage 1 and 
Satellite and Model Products are used for Stage 2 and beyond. Sparse Networks fall between these two 
stages. The above methodological elements 1-4 were engaged in preparation for the L4_C beta-release. 
However, all of these elements will be further engaged in preparation for the validated product release. 


6 PROCESS USED FOR BETA RELEASE 

In order to meet requirements for a November 2015 L4 product beta-release, the SMAP L4_C team 
generally confined the product assessment to the April 13-July 30, 2015 product record. April 13, 2015 
represents the beginning of the L4_C product series, while the designated end date was selected to allow 
sufficient time for analysis and preparation of the Beta Release Assessment Report. The team has been 
conducting frequent assessments of the L4_C operational product outputs and will continue to do this 
throughout the intensive Cal/Val phase and beyond. 

Frequent product performance and validation assessments were conducted over the initial 3.5 month 
record using tower eddy covariance measurement-based daily CO 2 flux observations from up to 30 
participating CVS tower sites. These comparisons involved spatially and temporally collocated daily 
tower observations and L4_C product outputs emphasizing NEE and GPP variables. Model performance 
was also evaluated against daily climatologies of the estimated carbon variables derived from long-term 
(2000-2013) MODIS fPAR and GEOS-5 (NRv4) surface meteorology records. The CVS comparisons 
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involved periodic teleconferences with the participating tower Pis to solicit local expertise in evaluating 
and interpreting product results in context with the tower observations and associated uncertainty. 

Model and product performance was also evaluated using more extensive historical daily tower 
observations from 228 globally distributed sites represented by the FLUXNET La Thuille synthesis 
dataset (Baldocchi 2008). Model estimated NEE RMSE performance was evaluated against the range of 
observed NEE variability over the global domain and within the major plant functional type (PFT) classes 
represented by multi-year tower observational records spanning a large global range of vegetation and 
climate conditions. 

The L4_C assessment activities included consistency checks against similar carbon variables 
available from other independent global data products, including the MODIS (Collection 5) MOD17A2 
GPP record (Zhao and Running, 2010) and the MPI-MTE global GPP record (Jung et al. 2010). A global 
monthly composited SIF (solar-induced canopy fluorescence) observation record derived from the ESA 
GOME-2 sensor (Joiner et al. 2013) was also used as a GPP proxy for evaluating estimated global 
patterns and latitudinal gradients in the L4_C GPP calculations. 

The L4_C product beta-release was determined on the basis of achieving a minimum set of 
satisfactory model and product performance metrics. These metrics involved: 1) demonstrations that the 
L4_C product outputs are consistent with known global and seasonal patterns, and that the magnitudes of 
estimated carbon fluxes are within realistic ranges for the major global PFT classes represented; 2) 
demonstrations that model performance is within design specifications, with no apparent model errors or 
anomalies. 


7 ASSESSMENTS 

7.1 Global Patterns and Features 

General global patterns and seasonal dynamics of the major L4_C product fields were evaluated prior to 
more robust quantitative assessments of product performance and accuracy. These qualitative assessments 
were used to evaluate whether the product outputs capture characteristic global patterns and seasonality as 
well as impacts from known climate anomalies, including major droughts, occurring within the 2015 
record. These qualitative assessments were also used to determine whether there were any apparent model 
errors or anomalies requiring more detailed model and product error diagnostics. The L4_C model 
processing is conducted at a daily time step and 1-km spatial resolution consistent with MODIS fPAR and 
land cover (PFT) inputs. The L4_C product outputs are posted to a 9-km resolution global EASE-grid 
(version 2) consistent with the SMAP Level 4 daily soil moisture (L4_SM) inputs. The primary daily 
product fields include vegetation gross primary production (GPP) and underlying environmental 
constraint (EC) metrics representing the proportional (%) reduction in estimated light-use efficiency 
(LUE) from potential conditions due to unfavorable environmental effects, including high vapor pressure 
deficits, cold daily minimum air temperatures, low soil moisture levels and frozen soil conditions 
(Kimball et al. 2012). The L4_C product fields also include heterotrophic respiration (Rh) and underlying 
soil moisture (Wmult) and soil temperature (Tmult) EC metrics. The primary carbon variable used for 
L4_C validation assessment is net ecosystem CCF exchange (NEE), which is computed as a residual 
difference between GPP and ecosystem respiration defined as the sum of Rh and estimated autotrophic 
respiration. 

The L4_C GPP outputs are presented in Figure 7.1 for two selected days in early and mid-spring of 
2015. These results also show the corresponding EC constraints on estimated LUE and GPP for each day. 
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Figure 7.1. L4_C daily product examples for April 13 th and May 25 th , 2015, showing vegetation gross primary 
production (GPP) and the EC metric, which is the proportion (%) of estimated light use efficiency relative to a 
potential maximum rate (LUE/LUE mx ) defined for optimal (non-limiting) environmental conditions. Grey areas 
denote barren land, permanent ice, open water and other areas outside of the model domain. 


These results depict the expected south-north progression of the Northern Hemisphere spring growing 
season onset and vegetation greening wave. Early spring conditions depicted by the April 1 3 map show 
low productivity (GPP) over the northern latitudes from widespread cold temperatures and associated 
strong EC restrictions. In contrast, much higher productivity levels occur over the northern latitudes in the 
May 25 th image due to relatively warm, moist conditions and associated relaxation of EC constraints on 
LUE. Other regional anomalies are also apparent in these images, including relatively low GPP levels and 
large EC restrictions over northern India resulting from a documented 2015 spring heat wave. Lower 
productivity areas are also apparent over the southwest USA and African Sahel due to seasonal drought- 
induced soil moisture restrictions on estimated productivity. These results also show characteristic higher 
productivity over the tropics, indicating successful model screening and substitution of missing and cloud 
contaminated MODIS fPAR inputs using alternative clear-sky values from the ancillary MODIS 8-day 
fPAR climatology in the L4_C preprocessor. 

Figure 7.2 depicts the L4_C model-estimated soil heterotrophic respiration (Rh) for the same two 
days in early to mid-spring 2015. The underlying cold temperature and low surface soil moisture EC 
constraints to the Rh calculation are also presented. These results show characteristically low respiration 
rates in early spring at higher latitudes prior to seasonal thawing, as indicated by strong Tmult reductions 
in the April 13 th image. In contrast, the Tmult constraints are relaxed after seasonal thawing with the 
arrival of warmer temperatures, resulting in widespread increases in Rh rates indicated in the May 25 th 
image. However, the potential increase in Rh is offset over many areas by surface soil moisture drying, 
including semi-arid areas of the southwest USA, southern Africa, and central Australia. 
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Figure 7.2. L4_C daily product examples for April 13 th and May 25 th , 2015, showing estimated soil heterotrophic 
respiration (Rh) and underlying surface soil temperature and soil moisture EC controls (Tmult, Wmult). The Tmult 
and Wmult EC metrics are dimensionless scalars ranging from 0 (fully constrained) to 100% (no constraint). Grey 
areas denote barren land, permanent ice, open water and other areas outside of the model domain. 


A selection of L4_C estimated daily NEE images extending from mid spring to early summer is 
presented in Figure 7.3. These maps show relatively large characteristic spatial heterogeneity in the sign 
and magnitude of the estimated carbon fluxes because NEE is a residual difference between much larger 
GPP and respiration fluxes. GPP and ecosystem respiration also tend to respond similarly to 
environmental changes, which can obscure more obvious environmental impacts affecting carbon 
source/sink activity. Nevertheless, the sequence of images depicts the seasonal transition from early 
spring carbon source activity in the northern latitudes to widespread carbon sink activity with the arrival 
of warmer temperatures and vegetation greening in summer. Generally stronger carbon sink activity is 
also depicted over Eurasia relative to North America during the summer season due to anomalous warm, 
dry conditions reported over northwest Canada and Alaska in 2015. Likewise, the southwest USA and 
California show widespread and persistent carbon source activity stemming from an extended and severe 
drought in these areas. 
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Figure 7.3. L4_C estimated NEE (g C m 2 d" 1 ) for four selected days between April 18 th and July 2 nd , 2015. Positive 
(negative) NEE fluxes denote net ecosystem carbon source (sink) activity. Black colors denote barren land, 
permanent ice, open water and other areas outside of the model domain. 


The L4_C surface soil organic carbon (SOC) field from the initial (April 2015) portion of the 
operational record is presented in Figure 7.4. The L4_C algorithms use a general three-pool soil 
decomposition model with cascading litter quality and associated soil decomposition rates encompassing 
variable turnover rates for labile, cellulosic and recalcitrant organic matter pools (Kimball et al. 2012, Yi 
et al. 2013). The SOC map in the figure represents the aggregation of these three soil carbon pools. These 
initial results largely represent carbon model spin-up conditions that reflect the daily climatological 
(2000-2013) forcing conditions from the GMAO SMAP Nature Run version 4 (NRv4) system used to 
initialize the L4_C model, including its SOC state at the beginning of the SMAP operational record. The 
observed SOC patterns generally capture the expected characteristic soil carbon distributions, including 
higher SOC stocks in cold northern boreal forest and tundra biomes estimated to hold more than half of 
the global soil carbon reservoir (Hugelius et al. 2014). The L4_C SOC map also shows relatively high soil 
carbon storage in temperate forest areas due to high forest productivity rates and cool, moist soils that 
promote soil carbon storage. Low SOC areas occur over drier climate zones, including desert areas in the 
southwest USA with generally low productivity levels, warmer climate conditions and associated low 
SOC accumulations. The L4_C results also show relatively low SOC levels in tropical forests; high 
characteristic GPP rates and associated litterfall inputs are generally offset by warm, moist soil conditions 
that promote rapid decomposition in the soil, so that the majority of terrestrial carbon storage in the 
tropics is in vegetation biomass (Baccini et al. 2012). 


13 






Figure 7.4. Estimated surface (< 10 cm depth) soil organic carbon (SOC, kg C m" 2 ) for April, 2015 from the SMAP 
L4_C operational record. The SOC estimates are derived at a 1-km spatial resolution during L4_C processing and 
posted to a 9-km resolution spatial grid. White areas denote barren land, permanent ice, open water and other areas 
outside of the model domain. 


7.2 Global Performance against Historical Tower Observations 

The L4_C validation assessment included comparisons of the model estimated daily carbon fluxes with 
ground-based observations of these variables from global sparse network tower eddy covariance CCL flux 
measurement sites as described in the SMAP Calibration/Validation Plan (Jackson et al. 2012). In situ 
data are critical in the assessment of the SMAP products. These comparisons provide for model 
performance and accuracy estimates and serve as a basis for modifying algorithms and/or parameters. A 
robust analysis requires many sites representing a diverse range of vegetation and climate conditions. The 
L4_C assessment included comparisons of L4_C estimates of daily NEE and GPP against spatially 
collocated, gap-filled daily observations of these parameters from 228 tower sites spanning the global 
domain and representing the major global PFT classes. The tower records were obtained from a larger set 
of tower site records from the FLUXNET La Thuile tower data synthesis (Baldocchi 2008). The tower 
sites enlisted for the comparisons were selected on the basis of being located within relatively 
homogenous terrain and land cover (PFT) areas defined from MODIS 1-km land cover data within 9km x 
9km windows centered over each tower site. The tower sites were also selected on the basis of having 
multi-year observational records with relatively well characterized observation uncertainty. 

The La Thuile tower record represents a global synthesis of FLUXNET daily tower observations 
where tower measurement records have been processed using consistent methods for temporal gap-filling 
of missing data, aggregation of daily carbon fluxes, and partitioning of NEE into component carbon 
fluxes. However, the La Thuile data record only extended to 2007, so that comparisons with the L4_C 
product outputs were spatially co-located but were not temporally consistent. Therefore, these 
comparisons focused on evaluating L4_C-based NEE and GPP performance in relation to historical daily 
means and temporal variability (SD) at the 228 globally representative tower sites. 

We conducted a spatial implementation of the L4_C error budget to map estimated ubRMSE 
performance for the NEE estimates over the global domain. Spatially explicit estimates of NEE ubRMSE 
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(g C m 2 yr' 1 ) were derived using a locally weighted forward model sensitivity analysis (Kimball et al. 
2014) driven by MODIS fPAR and GMAO SMAP NRv4 daily surface meteorology inputs. The NRv4 
data is derived using the same GEOS-5 land model underpinning the SMAP Level 4 Soil Moisture 
(L4 SM) product; these data were also used for calibration and initialization of the L4_C operational 
algorithms. The resulting global NEE error budget is presented in Figure 7.5. The figure depicts the 
tower sites used for the model performance assessment and includes a summary plot showing the mean 
and range of variability in the correlations between the global tower NEE observations and associated 
L4_C NEE estimates stratified according to PFT class. The number of tower sites (N) represented within 
each PFT class is shown at the top of the plot, while NEE ubRMSE estimates are summarized for all land 
areas within each PFT class and in relation to the observed RMSE and ubRMSE differences and NEE 
correlations with the tower observations representing each PFT class. These results indicate that 
approximately 66% and 83% of the global and northern (>45°) domains are within the targeted L4_C 
product performance threshold for NEE (ubRMSE<30 g C m 2 yr' 1 ). The estimated ubRMSE global 
performance is largely consistent with local assessments derived from tower NEE observations 
representing the major PFT classes. The magnitude of the NEE RMSE differences are proportional to 
ecosystem productivity (GPP) so that more productive sites such as croplands have generally greater 
RMSE levels than less productive (e.g. shrubland and grassland) areas. Thus, if we express the NEE 
RMSE as a proportion of the NEE flux, the estimated extent of meaningful NEE estimates (i.e., 

| ubRMSE/NEE | < 30%) increases to more than 80% of the global domain. Correlations between the 
model and tower NEE observations are generally greater in areas with larger characteristic seasonality, 
while EBF areas have lower correspondence largely due a smaller seasonal cycle in these predominantly 
tropical areas. Over northern land areas NEE ubRMSE levels are generally within the targeted accuracy 
threshold, except for some northern croplands and forests. Overall, these results indicate that the L4_C 
algorithms and beta-product are consistent with expected model design and performance specifications. 
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Figure 7.5. Estimated L4_C model and product performance for NEE in relation to in situ observations from 228 
tower sites representing the major global plant functional type (PFT) classes, including evergreen needleleaf forest 
(ENF), evergreen broadleaf forest (EBF), deciduous broadleaf forest (DBF), shrubland (SF1R), grassland (GRS), 
cereal (C3) croplands (CCR) and broadleaf (C4) croplands (BCR). The tower sites are depicted in the L4_C model- 
estimated NEE ubRMSE (g C m' 2 yr" 1 ) map (top). The lower plot includes a summary of mean model and tower 
NEE correlations and RMSE differences within each PFT class, with associated 25 th and 75 th percentiles of spatial 
variability; the number (N) of tower observation sites represented within each PFT class is denoted at the top of the 
plot. The estimated mean ubRMSE levels derived from the model sensitivity simulations (shown in upper map) for 
the tower pixel locations are also summarized within each PFT class in the plot. 


We also compared L4_C outputs from the initial operational record against daily mean GPP and 
NEE values from the historical tower observations; here L4_C outputs for the beta evaluation period (Apr 
13 - Jul 30, 2015) were compared against similar daily mean carbon fluxes from the 228 historical tower 
site records for the same (Apr 13 - Jul 30) seasonal period. The resulting spatial mean and variability 
(SD) in daily RMSE differences of the tower site comparisons within each PFT class are presented in 
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Figure 7.6. Both total and unbiased RMSE values are presented. These results are similar to the global 
annual performance summary described above (Figure 7.5), except that the performance assessment is 
conducted only for a limited (Apr to Jul) seasonal period using MODIS fPAR and SMAP-informed 
operational environmental inputs to the L4_C algorithms. These results indicate that the L4_C 
performance for the initial beta evaluation period is consistent with the algorithm design and targeted 
daily NEE accuracy threshold (ubRMSE<1.6 g C m 2 d" 1 ) for relatively less productive PFT classes 
characteristic of northern biomes, whereas croplands and deciduous broadleaf forests show higher RMSE 
levels consistent with characteristic higher productivity levels and NEE rates for these vegetation types. 
Surprisingly, the L4_C model and tower RMSE values are low for relatively productive tropical forests 
(EBF); however, the relatively few (14) towers representing this PFT class may not adequately represent 
global EBF diversity. The total RMSE levels are higher than the ubRMSE values due to systematic spatial 
and temporal bias in both model outputs and tower observations. The contributing sources and 
characteristics of model and tower bias are not well defined given the relatively short (Apr- Jul, 2015) 
operational record examined. Characterization of systematic bias is expected to improve with a longer 
operational record. The L4_C performance is also expected to improve with a longer SMAP observation 
record and associated calibration refinements, and reprocessing updates to the lower order sensor 
retrievals and L4 product outputs. 



Figure 7.6. Spatial mean and variability (SD) in daily RMSE differences between L4_C operational outputs for 
GPP and NEE and daily mean C fluxes derived from historical daily tower observations at the 228 FLUXNET sites 
for the L4_C beta evaluation period (April 13 to July 30). The comparison results are summarized within global PFT 
classes representing individual tower sites. Both total (black squares) and bias-adjusted (white squares) RMSE 
values are presented. The targeted accuracy threshold for NEE (1.6 g C m" 2 d" 1 ) is also shown (dashed line). 


7.3 Core Validation Sites 

The initial L4_C operational daily outputs (Apr 13 - Jul 30, 2015) were compared against in situ daily 
tower observations for up to 33 participating core tower validation sites. Unlike the historical site 
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comparisons described above, the CVS comparisons were both spatially consistent and temporally 
overlapping for the available 2015 record. The CVS comparisons are enabled by active participation from 
individual tower site principle investigators (Pis) in the SMAP L4_C Cal/Val process. The SMAP L4_C 
CVS sparse tower network is summarized in Table 7.1; the associated tower CVS locations are presented 
in Figure 7.7 along with the FLUXNET sites used for L4_C calibration and performance assessments 
(Section 7.2). 


Table 7.1. CVS sparse tower network used for intensive L4_C product assessments. 


Site 

2 PFT 

Lat 

Lon 

Location 

Full Name 

Fl-Sod 

ENF 

67.36 

26.64 

Finland 

FMI Sodankyla 

CA-Ojp 

ENF 

53.92 

-104.69 

Sask. CN 

BERMS Old Jack Pine 

CA-Obs 

ENF 

53.99 

-105.12 

Sask. CN 

BERMS Southern Old Black Spruce 

US-ICt 

SHR 

68.61 

-149.30 

AK, USA 

Imnavait Tussock 

US-ICh 

SHR 

68.61 

-149.30 

AK, USA 

Imnavait Heath 

US-ICs 

SHR 

68.61 

-149.31 

AK, USA 

Imnavait Wet Sedge 

US-BCr 

ENF 

64.70 

-148.32 

AK, USA 

Bonanza Creek Black Spruce 

US-BCb 

ENF 

64.70 

-148.32 

AK, USA 

Bonanza Creek Bog 

US-BCf 

ENF 

64.70 

-148.31 

AK, USA 

Bonanza Creek Fen 

US-PFa 

DBF 

45.95 

-90.27 

Wl, USA 

Park Falls 

US-FPe 

GRS 

48.31 

-105.10 

MT, USA 

Fort Peck 

US-Atq 

GRS 

70.47 

-157.41 

AK, USA 

Atqasuk 

US-lvo 

SHR 

68.49 

-155.75 

AK, USA 

Ivotuk 

US-Me2 

ENF 

44.45 

-121.56 

OR, USA 

Metolius Intermediate Pine 

US-Me3 

SHR 

44.32 

-121.61 

OR, USA 

Metolius Second Young Pine 

US-S02 

SHR 

33.37 

-116.62 

CA, USA 

Sky Oaks Old Stand 

US-S03 

SHR 

33.38 

-116.62 

CA, USA 

Sky Oaks Young Stand 

US-S04 

SHR 

33.38 

-116.64 

CA, USA 

Sky Oaks 

US-SRM 

SHR 

31.82 

-110.87 

AZ, USA 

Santa Rita Mesquite 

US-Wkg 

GRS 

31.74 

-109.94 

AZ, USA 

Walnut Gulch Kendall Grasslands 

US-Whs 

SHR 

31.74 

-110.05 

AZ, USA 

Walnut Gulch Lucky Hills Shrubland 

US-Ton 

SHR 

38.43 

-120.97 

CA, USA 

Tonzi Ranch 

US-Var 

SHR 

38.41 

-120.95 

CA, USA 

Vaira Ranch 

AU-Whr 

SHR -36.67 

145.03 

Australia 

Whroo 

AU-Rig 

CRP -36.66 

145.58 

Australia 

Riggs Creek 

AU-Yan 

CRP -34.99 

146.29 

Australia 

Yanco 

AU-Stp 

GRS -17.15 

133.35 

Australia 

Sturt Plains 

AU-Dry 

GRS -15.26 

132.37 

Australia 

Dry River 

AU-DaS 

GRS -14.16 

131.39 

Australia 

Daily River Savannah 

AU-How 

GRS -12.50 

131.15 

Australia 

Howard Springs 

AU-GWW 

SHR -30.19 

120.65 

Australia 

Great Western Woodlands 

AU-ASM 

SHR -22.28 

133.25 

Australia 

Alice Springs 
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AU-TTE SHR -22.29 133.64 Australia Ti Tree East 


'FLUXNET based tower site identifiers; 2 Tower PFT classes defined from a 1-km resolution MODIS (MOD12Q1) 
Type 5 global land cover map, consistent with L4_C processing. 



WAT ENF EBF DNF DBF SHR CRS CCR BCR URB ICE BAR 

Figure 7.7. Locations of the core (CVS) tower validation sites used for intensive L4_C product assessments; 
FLUXNET sites with historical tower records used for the L4_C model calibration and performance assessments are 
also shown in relation to global plant functional types summarized from the MODIS MOD12Q1 (Type 5) global 
land cover classification. 

Most of the CVS partners are providing near real-time daily tower observations, enabling temporally 
overlapping comparisons between the in situ tower observations and collocated SMAP L4_C operational 
product outputs. The CVS network spans 29 site locations and a broad range of climate and vegetation 
conditions; two of the sites (Imnavait, Bonanza Creek) include multiple towers sampling different 
vegetation communities within the larger sub-regions for a total of 33 participating CVS towers. Unlike 
the FLUXNET sites used for the L4_C global performance assessment (Section 7.2), the CVS towers 
may be located in spatially heterogeneous land cover areas with different PFT characteristics than the 
dominant vegetation class represented within the overlying 9-km resolution L4_C grid cell. Therefore, in 
addition to comparisons between tower observations and average daily model outputs for the 
corresponding overlying 9-km grid cell, the model-tower CV S assessment included comparisons between 
tower observations and (pre-aggregated, higher resolution) model output for the PFT within the cell that 
most closely represents the reported PFT within the local tower footprint. This is enabled by the 1-km 
resolution of the L4_C processing commensurate with the MODIS fPAR and PFT inputs; the sub-grid 
PFT outputs within each 9-km grid cell are preserved in the operational product outputs. 

The initial CVS comparisons associated with the L4_C beta-release were limited to the Apr 15 - Jul 
30, 2015, time period. The global distribution of CVS towers in both Northern and Southern Hemisphere 
locations allows for a relatively robust assessment of seasonality in the initial (Apr- Jul) record; 
environmental conditions were largely dormant at the northern high latitude tower sites in the post-launch 
early spring period, whereas the Southern Hemisphere tower sites were experiencing austral late summer 
and early fall conditions. Due to the CVS requirements for frequent (weekly) tower data delivery updates, 
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uncertainty in the CVS tower records is expected to be larger than would otherwise be expected from 
longer and more refined science data quality records. Local assessments of tower data quality were 
provided by many of the tower Pis as part of their data deliveries; the criteria and structure of the data 
quality metrics provided varied across the different tower sites, but they all give at least some qualitative 
indication of the reliability of the tower observations. 

The L4_C performance assessment against CVS daily tower observations for NEE and GPP is 
presented in Figure 7.8. These results represent CVS sites having available tower observations within the 
beta evaluation period and are a subset of the full CVS network listed in Table 7.1. The reported metrics 
for each site include RMSE and mean residual bias computed between the tower observations and 
collocated model outputs representing daily mean carbon fluxes within the overlying 9-km resolution 
model grid cell; RMSE, bias and R 2 correspondence metrics are also derived for the model sub-grid PFT 
mean that is most similar to the reported local tower footprint PFT. The estimated bias-adjusted RMSE 
metrics are also presented. The initial CVS comparisons show several apparent tower site outliers, 
including Metolius, Bonanza Creek and Howard Springs. However, these outliers were largely traced to 
incorrect processing of the tower observations rather than to model error. Other outliers were caused by 
PFT mismatches between the local (~l-km 2 ) tower footprint observations and overlying L4_C model grid 
cells. For example, the Howard Springs tower footprint is characterized as woody savanna with a 
grass/shrub understory and Eucalyptus forest overstory. The MODIS MOD12Q1 Type 5 global land 
cover classification used for L4_C processing doesn’t include a woody savanna PFT category; instead, 
the 9-km grid cell overlying the Howard Springs site is predominantly characterized as GRS, with smaller 
proportional DBF representation; neither of these PFT categories provides a close functional match with 
the local Howard Springs vegetation, leading to larger model-tower differences. Despite the anomalous 
outlier sites described above, the model-tower comparison results show generally favorable performance, 
with L4_C derived NEE outputs near the targeted RMSE level, including both total and bias-adjusted 
error metrics. 

The initial CVS comparisons reveal several challenges associated with maintaining robust quality 
control over a diverse set of tower observations and participating Cal/Val partner teams, despite efforts to 
standardize data formatting and submission of frequent tower observation updates. The CVS tower 
observations have larger uncertainty relative to longer historical data records such as the La Thuile 
FLUXNET record (Section 7.2). The larger tower observation uncertainty is due to multiple factors, 
including project requirements for frequent updating of tower data, which restricts the amount of time 
available for detailed post-processing analysis and data quality checks. The relatively short (~3.5 mos) 
period of record of the initial CVS observations also contributes to larger uncertainty associated with gap- 
filling of missing data and estimation of component carbon fluxes (e.g. GPP) from the tower NEE 
measurements. While many of the tower partners have contributed tower data quality metrics, these data 
have not yet been incorporated into the CVS analysis. The CVS comparisons will continue to be updated 
and refined using longer data records acquired during the post-beta release period. The relative quality 
and reliability of the CVS comparisons is expected to improve with longer data records incorporating a 
full annual cycle and with further refinements to the L4_C Cal/Val matchup tools. 
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Tower Composite Validation Metrics for Gross PrimaryProductivity (GPP) 
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Figure 7.8. Summary of initial CVS comparisons between daily tower observations and L4_C daily product outputs 
for GPP and NEE from the April 15 - July 30, 2015 operational record. The reported metrics include R 2 
correspondence, RMSE and bias between tower observations and L4_C spatial mean daily conditions within the 
overlying 9-km resolution product grid cell (9km R 2 , 9km RMSE, 9km bias); RMSE, bias and R 2 correspondence 
between the tower observations and the most similar PFT mean within the 9-km grid cell (PFT RMSE, PFT bias, 
PFT R 2 ); and the associated bias-adjusted RMSE values (aRMSE). The targeted daily NEE RMSE threshold for the 
L4_C product (1.6 g C m' 2 d" 1 ) is also shown. 
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7.4 Consistency with Other Global Carbon Products 


The L4_C product outputs were compared against similar variables from other available global carbon 
products. The objective of these comparisons was to assess and document the general consistency of 
selected L4_C operational product fields in relation to similar variables from other global benchmark 
datasets commonly used by the community. The global data products examined are publicly available and 
include: 


1) The NASA EOS MODIS MOD17A2 (C5) operational GPP product, with 1-km 
resolution and 8-day temporal fidelity (Zhao and Running, 2010); 

2) The Max Planck Institute’s Model Tree Ensemble (MPI-MTE)-based global tower 
observation upscaling of monthly GPP from the La Thuile FLUXNET synthesis 
observational record (Jung et al. 2010); 

3) Solar-Induced canopy Fluorescence (SIF) observations from the ESA GOME-2 (Global 
Ozone Mapping Experiment) satellite sensor (Joiner et al. 2013); 

4) Global soil organic carbon (SOC) inventory records from the International Geosphere 
Biosphere Program Data Information System (IGBP-DIS; Global Soil Data Task Group 
2000) and the Northern Circumpolar Soil Carbon Database (NCSDC; Hugelius et al. 
2014). 

Global consistency checks between the selected L4_C outputs and these datasets involved comparing 
mean latitudinal distributions of average product fields for the April - July, 2015, period against average 
conditions for the similar period derived from the other historical data records. The MODIS MOD17A2 
and MPI-MTE GPP records used for this analysis extended from 2000 to 2014 and from 2000 to 201 1, 
respectively. The GOME-2 SIF record used for this study represented composited mean monthly 
observations extending from 2007-2013. Differences were computed between the L4_C GPP maps and 
the MOD 1 7 and MPI-MTE GPP maps to evaluate the global distribution of differences in estimated 
productivity. The SIF retrieval from GOME-2 is related to LUE and photosynthesis and was used as an 
observational proxy for GPP. 

The available SOC inventory records represent static maps extending over global (IGBP-DIS) and 
northern (NCSCD) domains. Surface (<10cm depth) SOC stocks were estimated as a fixed proportion 
(33.33%) of the total soil profile (0-100 cm depth) SOC stock records. These data were compared against 
the initial L4_C SOC product outputs from April, 2015. The SOC records were evaluated by comparing 
latitudinal means and spatial SD ranges. A global SOC difference map was also computed between the 
L4_C and IGBP-DIS records to evaluate the spatial pattern of SOC differences. Detailed summaries of 
these comparisons are provided in the following sub-sections. 

7.4.1 MODIS MODI 7 GPP 

The L4_C daily GPP product fields were averaged over the April to July, 2015, period and compared 
against the MOD17A2 GPP climatological means for the same months, but derived from the long-term 
(2000-2014) MODIS record. The averaged MODIS GPP data was re-projected from a 5-km resolution 
geographic projection to the 9-km resolution global EASE-grid (V.2) format of the L4_C product using 
nearest-neighbor resampling. A GPP difference map was then computed by grid cell-wise subtraction of 
MOD 17 values from collocated L4_C values. The distributions of the GPP spatial means and ±1 SD 
spatial variations were also computed along 0.05 degree latitudinal bins and compared for relative 
consistency between the MOD 17 and L4_C datasets. These results are presented in Figure 7.9. 
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Figure 7.9. Comparison of mean daily GPP outputs (April to July; g C m" 2 d" 1 ) from the SMAP L4_C and MODIS 
MOD17A2 (C5) global data products. The L4_C outputs are derived from the data record for 2015, while the 
MOD17A2 results represent climatological means derived from the long-term (2000-2014) MODIS record. The 
distributions of GPP spatial means and ±1 SD spatial variations are summarized within 0.05 degree latitudinal bins 
(top); the GPP difference map between the SMAP and MOD 17 results is also shown (bottom) along with the 
relative mean latitudinal distribution of the GPP difference (grey shading along Y-2 axis). White areas in the 
difference map represent barren land, permanent ice, open water bodies and other areas outside of the L4_C product 
domain. 


The L4_C GPP product fields for the initial (April - July, 2015) operational record examined show a 
global productivity distribution similar to that in the MODIS MOD17A2 GPP record. The resulting 
productivity patterns show generally similar magnitudes, latitudinal distributions and spatial variability. 
However, there are also significant regional differences that can be attributed to one or more factors, 
including differences in model calibrations and underlying environmental controls on estimated GPP, and 
differences between 2015 climate (the year used for the L4_C record) and that for 2000-2014 (used for 
the long-term MODIS climatology). Areas with relatively lower L4_C GPP levels (red colors) include 
northern boreal forests, the southwest USA, central Australia and the sub-tropics. These areas have 
generally significant seasonal dry periods where plant-available soil moisture limitations reduce 
productivity from potential conditions. The lower L4_C GPP levels in these areas reflect the impact of 


23 



including additional soil moisture constraints to productivity in the L4_C algorithms relative to the use of 
atmospheric VPD as the sole moisture constraint to productivity in the MOD 17 LUE algorithm. Other 
areas with relatively lower L4_C productivity levels reflect documented dry climate anomalies in the 
2015 record relative to the longer term climate conditions represented by the MOD 17 record. These 
climate anomalies include ongoing drought conditions over California and the southwest USA and 
relatively warm, dry spring and summer 2015 conditions over boreal Alaska, the north central USA and 
Canada. 

The GPP difference maps also show many areas with higher L4_C productivity levels (green colors) 
relative to the MOD 17 GPP results. The L4_C results show generally higher productivity levels over the 
temperate zone, particularly over intensive cropland areas, including eastern China, India, and the central 
USA. These differences are largely attributed to improved calibration of C3 and C4 croplands in the 
L4_C LUE model relative to the MOD 17 LUE model (Turner et al. 2006, Madani et al. 2014). The 
improved L4_C calibration in these areas partially reflects the use of a greater number of cropland tower 
sites in the model BPLUT calibration and the enhanced MODIS fPAR quality screening used during 
L4_C processing (Kimball et al. 2012). Other areas with relatively higher L4_C productivity rates reflect 
the influence of regional climate variations favoring greater productivity levels in 2015 relative to long- 
term average conditions represented by the MOD 17 record; these areas include relatively higher 
productivity in response to an early thaw season over northern Alaskan tundra. 

7.4.2 MPI-MTEGPP 

The L4_C mean daily GPP product fields from the April to July, 2015, period were compared against the 
MPI-MTE GPP record averaged over the same months, but derived from the longer MPI-MTE monthly 
record extending from 2000-2011 (Jung et al. 2010). The MPI record is derived using a Model Tree 
Ensemble (MTE)-based machine learning algorithm and empirical upscaling of tower-based daily GPP 
observations from the global FLUXNET data archive. The MTE spatial upscaling approach also uses 29 
explanatory geospatial variables, including monthly fPAR from the SeaWiFS sensor. The monthly MPI 
GPP data was re-projected from a 0.5 degree spatial resolution and geographic projection to the 9-km 
resolution global EASE-grid (V.2) format of the L4_C product using nearest-neighbor resampling. A 
GPP difference map was then computed by grid cell-wise subtraction of MPI values from collocated 
L4_C values. The distributions of the GPP spatial means and ±1 SD spatial variations were also computed 
along 0.05 degree latitudinal bins and compared for relative consistency between the MPI and L4_C 
datasets. These results are presented in Figure 7.10. 

Consistent with the MOD 17 comparison described above, the L4_C and MTE results show generally 
similar GPP magnitudes and latitudinal mean distributions, with generally higher producivity levels in the 
tropics and temperate zones and lower productivity levels at higher latitudes. However, the MTE results 
show lower GPP spatial heterogeneity as indicated by relatively narrow SD in the latitudinal distribution 
plots, especially over the lower latitudes and Southern Hemisphere. The narrower MTE range of 
variability is attributed to the limited number of global in situ tower sites used in the MTE spatial 
extrapolations; the global distribution of tower network sites is particularly sparse over the tropics and 
Southern Hemisphere land areas (Schimel et al. 2015). 

As with the previous MOD 17 comparison, the L4_C results indicate generally lower productivity 
than the MTE results over drier climate areas, including boreal forest and subtropical zones. These 
differences are attributed to the impact of dynamic soil moisture-related water supply constraints used in 
the L4_C GPP calculations; the MTE results are based on generalized precipitation and potential 
evapotranspiration indices used to define moisture-related impacts on productivity spatial patterns. The 
L4_C results also show generally lower GPP levels over the temperate zone, including tropical forest and 
cropland areas. The L4_C GPP levels in these areas are generally intermediate between MOD 17 and 
MTE. 
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Figure 7.10. Comparison of mean daily GPP outputs (April to July; g C m' 2 d" 1 ) from the SMAP L4_C and MPI- 
MTE global data products. The L4_C outputs are derived from the initial data record for 2015, while the MTE 
results represent climatological means derived from a long-term (2000-2011) monthly record. The distributions of 
GPP spatial means and ±1 SD spatial variations are summarized within 0.05 degree latitudinal bins (top); the GPP 
difference map between the SMAP and MTE results is also shown (bottom) along with the relative mean latitudinal 
distribution of the GPP differences (grey shading along Y-2 axis). White areas in the difference map represent 
barren land, permanent ice, open water bodies and other areas outside of the L4_C product domain. 


7.4.3 GOME-2 SIF 

The L4_C mean daily GPP product fields from the April to July, 2015, period were compared against 
satellite-based SIF observations averaged over the same months, but derived from the longer (2007-2013) 
GOME-2 record (Joiner et al. 2013). The SIF data from the ESA GOME-2 sensor are available as 
composited monthly means at a 0.5 degree spatial resolution geographic projection format. The SIF data 
were re -projected to the 9-km resolution global EASE-grid (V.2) format of the L4_C product using 
nearest- neighbor resampling. The distributions of GPP and SIF spatial means and ±1 SD spatial 
variations were computed along 0.05 degree latitudinal bins and compared for relative consistency 
between the L4_C and GOME-2 datasets. SIF is proportional to canopy absorbed photosynthetically 
active radiation (APAR) and photosynthesis, but the relationship between SIF (mW m 2 sr' 1 nm' 1 ) and 
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GPP (g C m 2 d' 1 ) can vary according to vegetation type and environmental conditions (Porcar-Castell et 
al. 2014). Therefore, the SIF and GPP results are presented side-by-side in Figure 7.11 to allow a 
qualitative assessment of the inferred productivity patterns. 
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Figure 7.11. Comparison of mean daily GPP outputs (April to July; g C m" 2 d" 1 ) from the SMAP L4_C product 
against mean SIF (mW m" 2 sr" 1 nm" 1 ) observations from GOME-2 for the same months. Here SIF is used as a proxy 
for GPP even though the relationship between GPP and SIF can vary, particularly under environmental stress. The 
L4_C outputs are derived from the initial data record for 2015, while the SIF results represent climatological means 
derived from a long-term (2007-2013) monthly record. The distributions of GPP and SIF spatial means and ±1 SD 
spatial variations are summarized within 0.05 degree latitudinal bins (top), while the corresponding mean GPP and 
SIF global maps are also shown (bottom). White areas in the maps represent barren land, permanent ice, open water 
bodies and other areas outside of the L4_C product domain. 


Both the GOME-2 SIF and L4_C GPP records indicate similar global productivity distributions, 
including higher productivity rates in the tropical and temperate zones and lower productivity levels at 
higher latitudes. The spatial productivity patterns are also similar between the two datasets, including 
generally higher productivity levels over tropical and temperate forests and cropland areas, and lower 
productivity rates in drier climate zones. There are also, however, notable differences between the two 
products. The SIF data indicate generally higher productivity rates over croplands, which may be due to 
incorrect model parameterization and underestimation of optimal light use efficiency levels for 
intensively managed cropland vegetation (Guanter et al. 2013, Madani et al. 2014). Similar 
underestimation of cropland productivity levels is also apparent in the MOD 1 7 and MPI-MTE GPP 
records (e.g. Figures 7.9 and 7.10). The L4_C GPP results also indicate relatively lower productivity 
levels over temperate and tropical forests. Other differences in the productivity patterns are partially 
attributed to the coarser spatial footprint and temporal compositing of the GOME-2 observations relative 
to the finer (1-9 km) resolution of the L4_C processing, and differences between the shorter 2015 L4_C 
sampling period relative to average conditions defined from the long-term (2007-2013) GOME-2 record. 
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7. 4. 4 Soil Inventory Records 


The L4_C SOC stock estimates from the initial April 2015 record were compared against available static 
SOC inventory records extending over global (IGBP-DIS) and northern (NCSCD) domains. The L4_C 
SOC fields are computed on a daily basis as the residual difference between estimated litterfall inputs 
from vegetation net primary production, and soil heterotrophic respiration losses from the decomposition 
of soil organic matter. The L4_C SOC estimates are approximately representative of upper soil (<10 cm 
depth) conditions determined from the surface soil moisture (Wmult) and temperature (Tmult) constraints 
to the Ri, calculations (Kimball et al. 2012). Therefore, surface SOC stocks were estimated from the 
IGBP and NCSCD global inventory data as a fixed proportion (33.33%) of the reported total soil profile 
(0-100 cm depth) SOC stock records, though the actual proportion can vary from approximately 29 to 57 
percent or more (Jobbagy et al. 2000). These data were then compared against the initial L4_C SOC 
product outputs from April, 2015. The IGBP and NCSCD maps were re-projected from a 0.5 degree 
spatial resolution and geographic projection to the 9-km resolution global EASE-grid (V.2) format of the 
L4_C product using nearest-neighbor resampling. A global SOC difference map was then computed by 
grid cell-wise subtraction of the estimated IGBP surface SOC values from collocated L4_C SOC values. 
The distributions of the SOC spatial means and ±1 SD spatial variations were also computed along 0.05 
degree latitudinal bins and compared for relative consistency between the inventory records and L4_C 
data. The NCSCD record is only available for the northern land areas, while the IGBP record extends 
over the global domain. 

The resulting L4_C and IGBP global SOC difference map and the mean latitudinal distributions of 
the different SOC estimates are presented in Figure 7.12. The latitudinal SOC distribution indicated by 
the L4_C record is similar to the soil inventory records, though the L4_C results are generally at the lower 
range of the reported inventory data. The L4_C results show a similar increase in SOC levels at higher 
latitudes consistent with characteristic soil carbon storage increases in colder boreal and tundra soils. The 
difference map between the L4_C and IGBP results indicates larger model underestimation in northern 
boreal forest and tundra areas known to contain a large portion of the global soil carbon stock stored in 
permafrost soils, which have accumulated carbon at relatively slow rates over millennia. The apparent 
L4_C SOC underestimation in these areas may be due to the recent (2000-2013) NRv4 daily climate 
record used for model SOC initialization; the observed SOC stocks reflect long-term climate conditions. 

Surface SOC Difference (L4C - IGBP) Mean SOC 



Figure 7.12. Difference map between L4_C and IGBP estimated surface SOC stocks (kg C m" 2 ). The mean 
latitudinal SOC distributions and ±1 SD spatial variations (shaded) from the L4_C, IGBP and NCSCD records are 
also shown. 
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The L4_C results and IGBP inventory record both show generally lower soil carbon stocks in the 
temperate and tropical zones, though the L4_C results show lower SOC levels than the IGBP record in 
the tropics, including the African Congo and portions of tropical Southeast Asia. These relatively low 
L4_C SOC estimates are due to high estimated soil decomposition and Rh rates in warm, moist tropical 
climate conditions, with minimal constraints from low soil moisture conditions (e.g. Figure 7.2). The 
apparent SOC differences may also reflect regional bias in the GEOS-5 land model-derived daily climate 
inputs used in the L4_C calculations (Yi et al. 201 1). 

7.5 Summary 

This report provides an assessment and documentation of the SMAP Level 4 Carbon (L4 C) beta 
product. Methods used to ascertain L4_C beta-product quality and performance included: 1) qualitative 
evaluations of the product fields for representing expected characteristic spatial and seasonal patterns; 2) 
comparisons of daily product outputs with tower eddy covariance measurement-based daily carbon (CCL) 
flux observations from up to 30 core (CVS) tower sites; 3) comparisons of L4_C daily carbon flux 
estimates against a larger set of historical tower observations from 228 globally distributed FLUXNET 
sites; and 4) consistency checks of L4_C product fields against other synergistic global carbon products. 
This L4_C beta release assessment meets or exceeds CEOS Stage 1 validation criteria based on a limited 
set of core validation sites. The activities described in this report also satisfy criteria for Stage 2 validation 
by expanding to regional and global assessments that involve a diverse set of independent observations. 

The primary methods and metrics used for the L4_C Cal/Val assessment included comparisons of 
collocated time series of in situ tower observations and L4_C daily outputs. Comparisons involving CVS 
sites are both spatially and temporally consistent, while comparisons using the more extensive FLUXNET 
tower site records are spatially but not temporally consistent as they involve product evaluations against 
historical tower observations. Other methods employed for the evaluations included qualitative 
comparisons of latitudinal means and spatial patterns between L4_C outputs and other satellite, inventory 
and model -based products, including the NASA EOS MODIS MOD 17 GPP record, the MPI-MTE global 
tower observation upscaled GPP record, solar-induced canopy fluorescence (SIF) observations from the 
ESA GOME-2 satellite, and surface soil organic carbon (SOC) stock estimates from global soil inventory 
records. 

Based on these initial assessments, the beta-release L4_C product demonstrates a level of 
performance and accuracy consistent with the algorithm and product design specifications. The L4_C 
beta product shows expected characteristic global patterns and seasonality in estimated carbon fluxes and 
SOC stocks, with no evidence of obvious artifacts or errors in algorithm performance or formatting. The 
initial L4_C global performance assessment indicates that the beta product meets targeted accuracy 
requirements for NEE (RMSE<30 g C m 2 y' 1 or 1.6 g C m 2 d" 1 ) over approximately 66% and 83% of 
global and northern domains, respectively; these results are consistent with the initial CVS and global 
sparse network tower comparisons. The major L4_C product fields are also generally consistent with 
similar variables obtained from a diverse set of global benchmark environmental data records. The L4_C 
beta release is suitable for public distribution and utilization by the larger science and application 
communities. This beta release also presents an opportunity to enable users to gain familiarity with the 
parameters and data formats of the product prior to release of the L4_C validated product. The L4_C 
product performance and accuracy is expected to improve with continuing operations, Cal/Val 
refinements and reprocessing updates in preparation for the validated product release. 
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8 OUTLOOK AND PLAN FOR VALIDATED RELEASE 


For the beta release the L4_C team Stage 1 and Stage 2 (global assessment) activities have been 
established and are relatively mature. These Cal/Val program activities will continue toward validated 
release, including analyses of longer data records and updates from planned calibration refinements. 

These activities will continue through all Cal/Val stages over the SMAP mission life span. This report 
notes several limitations in the beta-release version of the L4_C product, including the use of GEOS-5 
surface temperatures rather than SMAP microwave sensor-defined freeze-thaw (FT) constraints on the 
estimated carbon fluxes. These limitations will be addressed prior to the L4_C validated release. The 
validated release will also include refinements gained from more extensive validation activities involving 
a longer operational data record and associated calibration improvements to lower order (L3 and L4) 
inputs to the L4_C algorithms. The validated release will include additional algorithm and product 
refinements gained from more extensive model sensitivity and validation studies involving several 
intensive field experiments. These activities will also enable a more comprehensive analysis and 
quantification of product bias, including systematic and random error components, spatial and seasonal 
patterns, and error sources. Some issues that should be considered between the beta and validated release 
include the following: 

• Moving toward a Stage 2+ validated product. The L4_C beta release is limited by a relatively 
short (April - July, 2015) period of record assessment. By the time of the planned validated 
product release in Summer 2016, there will be more than a year of SMAP observations spanning 
a complete global annual cycle. With enhanced inter-comparisons described below, the L4_C 
validation level should exceed CEOS validation Stage 2 and be within Stage 3 criteria. 

• Utilization of SMAP FT inputs. The L4_C beta release defines daily frozen temperature controls 
to ecosystem productivity using relatively coarse resolution GEOS-5 land model-based surface 
temperatures. This potentially limits the ability of the L4_C product to address SMAP carbon 
science objectives to quantify frozen temperature constraints to productivity and improve 
understanding of the boreal carbon sink. The L4_C validated release will include SMAP 
microwave sensor-based freeze-thaw (FT) constraints to GPP, with enhanced L-band sensitivity 
to landscape FT dynamics. These activities will include the utilization of more refined SMAP FT 
global product inputs with stable performance and demonstrated accuracy. The L4_C validated 
release should therefore enable a more comprehensive assessment and attainment of the SMAP 
carbon cycle science objectives. 

• More comprehensive algorithm sensitivity studies. Detailed L4_C algorithm sensitivity studies 
will be conducted to evaluate the global impact of SMAP observations on the model-estimated 
carbon fluxes. These simulations will be conducted by evaluating alternative model simulations 
derived with and without SMAP observation-informed inputs, including FT and SM inputs and 
their individual and combined impacts. These simulations will clarify the improved assessment 
and understanding gained through the SMAP observations on the estimated L4_C product 
variables spanning a global domain and complete annual cycle. These simulations will also utilize 
lower order (L3 and L4) SMAP inputs that are relatively mature (post beta release) enabling a 
more robust assessment of SMAP science impacts. 

• Refined CVS assessment. The methodology and outcome of the CVS comparisons should be 
relatively mature by the L4_C validated product stage. Milestones to be completed include 
acquiring and analyzing a longer set of tower observations spanning a full annual cycle for all 
participating CVS towers. The tower observation uncertainty is expected to be better 
characterized and incorporated into the L4_C validation assessment. The CVS matchup software 
used for the comparisons is also expected to be mature, including the utilization of temporally 
dynamic data quality metrics for both tower and product outputs. 
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• More extensive satellite data comparisons. A variety of global carbon products have been used 
for evaluating the general quality and performance of the L4_C beta release. These activities will 
also be incorporated into the validated assessment but will involve a longer data record spanning 
a full annual cycle. Additional satellite environmental data records will also be evaluated, 
including overlapping SIF observations from the NASA Orbiting Carbon Observatory (OCO-2). 

• Incorporating Field Campaign datasets for algorithm assessment and refinement. Several field 
campaigns will be completed in 2015, including the Australian SMAPEx and Arizona 
SMAPVEX15 campaigns. Other airborne field campaigns have recently been completed (e.g., 
the NASA AirMOSS and CARVE campaigns) that provide extensive ecological data records 
suitable for L4_C product evaluations. These data will be analyzed in the context of detailed 
L4_C algorithm sensitivity assessments and product comparisons in preparation for the validated 
release. These comparisons will focus on spatial scaling assessments, model sensitivity analyses, 
and performance evaluations involving a large global range of climate and vegetation conditions. 
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